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SYSTEM AND METHOD FOR IMPROVED SCALABILITY SUPPORT IN 

MPEG-2 SYSTEMS 

The present invention relates generally to scalable video coding systems, and 
5 more particularly, to a flexible and cost effective heterogeneous layered video decoding 
technique that allows the video encoding/decoding format to be independently selected 
per layer. 

In recent years, digital video storage has been introduced on various media, such 
as hard disks and optical discs (e.g. DVD+RW). From a consumer point of view, tiie 

10 amount of recording time should be fixed or at least guaranteed. With current 

compression schemes Has is achieved by controlling the quantize parameter. One 
drawback, however, is that the bit rate required for an artifact free picture greatly depends 
on the input sequence. For example, if the selected (average) bit rate is too low for an 
input sequence, it will result in coding artifacts like blocking as can be demonstrated 

15 using an appropriate metric. These artifacts could have been avoided if the sequence was 
compressed at a lower resolution. Although this is possible witii current standards like 
MPEG, it is limited to only static sequences and in abrupt discrete steps (SDTV, 1/2D1, 
CIF). Such abrupt changes in resolution can be quite annoying for the viewer. 

Apart from storage applications, tihie problem of occurring artifacts can also be 

20 observed in wireless video connections, e.g. using IEEE802. 1 lb, were the available bit 
rate is not always sufficient to cany the full SDTV resolution. 

What is needed, therefore, is a method which allows for dynamically adapted 
video resolution compression that can make use of existing compression standards like 
MPEG as building blocks. 

25 The present invention addresses the foregoing need by providing a heterogeneous 

layered video decoding system and associated method that uses only generic MPEG- 
2/4/AVC decoders to decode an MPEG-2/4/AVC compliant stream. In one embodiment, 
this is achieved by utilizing a parameter list to be transmitted along with the MPEG- 
2/4/AVC compliant stream that independently defines for each layer, how the particular 

30 layer is to be decoded. The parameter list may define for each layer, values to determine: 
(1) whether the particular layer is be scaled up, down or not at all (2) whether DC 
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compression is to be applied to the layer, (3) the type of stream (e.g. JVIPEG-2/4) that 

defines the layer, (4) the FIR coefficients, and (5) constant gains in the sub-band. The 

parameter values are preferably multiplexed along with the encoded signal to allow the 

decoder to interpret the parameter values and decode accordingly. 
5 In one aspect, in the case where there are more than two enhancement layers, a 

wide range of quality levels may be defined. For each quality level, flie encoder can 

transmit a separate parameter list. For example, for a four layer video stream including a 

base layer and three enhancement layers, a first parameter list could be constructed to 

define a combination of the base layer BS with both enhancement layers ESI and ES2. A 
10 second parameter list could be constructed to define a combination of the base layer BS 

with the second and fourth enhancement layers (BS + ES2 + ES4). Other combinations 

should be apparent to the reader. All of the combinations of interest to a user may be 

simultaneously transmitted as elements of parameter list. 

The foregoing features of the present mvention will become more readily 
15 apparent and may be understood by referring to the following detailed description of an 

illustrative embodiment of the present invention, taken in conjimction with the 

accompanying drawings, where: 

FIG. 1 is a block schematic representation for illustrating the principles of 

scalable coding (spatial scalability); 
20 FIG. 2 is a block schematic representation of a spatial scalable video encoder 

according to one embodiment of the invention; 

FIG. 3 is a block schematic representation of a spatial scalable video decoder for 

decoding the encoded signals processed by the layered encoder Fig. 2; 

FIG. 4 illustrates one example of a parameter list that would be broadcast over a 
25 communication channel as supplemental information to inform a decoder as to how to 

combine the various streams (e.g., Layl, Lay2) of a transport stream to output a single 

decoded video stream; 

FIG. 5 illustrates another example of a parameter list that would be broadcast over 

a communication channel as supplemental information to inform a decoder as to how to 
30 combine the various streams (e.g., Layl, Lay2) of a transport stream to output a single 

decoded video stream; 
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FIG. 6 illustrates a further example of a parameter list that would be broadcast 
over a communication channel as supplemental information to inform a decoder as to 
how to combine the various streams (e.g.. Lay 1, Lay2) of a transport stream to output a 
single decoded video stream; 
5 FIG. 7 is a block schematic representation of a spatial scalable video decoder for 

decoding the encoded signals in accordance with the parameter list of Fig. 6; 

FIG. 8 illustrates a further example of a parameter list that would be broadcast 
over a commimication channel as supplemental information to inform a decoder as to 
how to combine the various streams (e.g.. Lay 1, Lay2) of a transport stream to output a 
10 single decoded video stream; and 

FIG. 9 is a block schematic representation of a spatial scalable video decoder for 
decoding the encoded signals in accordance with the parameter list of Fig. 8. 

Although the following detailed description contains many specifics for the 

15 purpose of illustration, one of ordinary skill in the art will appreciate that many variations 
and alterations to the following description are within the scope of the inventioiL 
Accordingly, the following preferred embodiment of the mvention is set forth without 
any loss of generality to, and without imposing limitations upon, the claimed invention. 
The invention provides a number of specific advantages over prior art systems. 

20 Specifically, the system and method of the invention provides for flexible and cost 

eflfective scalability through the use of generic MPEG-2/4/AVC decoders at each layer 
instead of decoders specifically designed for scalable systems. A further advantage of the 
invention is that it allows for trade-offs between complexity and eflHciency. For example, 
the base layer may employ a sophisticated base layer AVC codec, while one or more 

25 enhancement layers may \ise an MPEG-2 codec that is half as complex as a full AVC 
codec but only slightly less efficient. A still further advantage is that die system and 
method of the invention allows for seamless migration fi*om one standard to another. In 
other words, presently the majority of broadcasters broadcast using the MPEG 
compression standard. As newer compression standards emerge, the same signal qxiality 

30 can be achieved at a lower bit rate. The present invention allows the base layer to be 
transmitted using the MPEG compression standard and as equipment upgrades are 
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realized, the enhancement layers can be transmitted using the newer compression 
standards. The migration can occur gradually as the system of the invention can be 
adapted to any qualily of service (QOS) configurations defined by the user. 

A further advantage of providing heterogeneous layered video support is 
5 illustrated in the case where a user is mitially only decoding a video stream in the base 
layer in a set top box, for example. Assume at some later point in time that the user also 
desires to use the Internet as an overlay. That is, in addition to supporting the video 
coding at llie base layer, the decoding of the video stream at the base layer remains fully 
supported by simply utilizing a lower quality of service (Qos) at the enhancement 

10 layer(s). Another advantage is a cost savings which may be realized when using generic 
MPEG-2/4/AVC decoders as compared with full qualily advanced (complex) codecs. A 
further advantage is low power (base layer only) decoding for battery operated, portable 
or mobile, equipment; quality of service (Qos) with respect to the transport of bits, and 
quality of service with respect to the cycle budget of a DSP. 

IS A brief review of general scalable coding (spatial scalability) is first provided. 

Many applications desire the capability to transmit and receive video at a variety of 
resolutions and/or qualities. One method to achieve this is with scalable or layered 
coding, which is the process of encoding video into an independent base layer and one or 
more dependent enhancement layers. This allows some decoders to decode the base layer 

20 to receive basic video and other decoders to decode enhancement layers in addition to the 
base layer to achieve higher temporal resolution, spatial resolution, and/or video quality. 

The general concept of scalability is illustrated in FIG. 1 for a codec with two 
layers. Note that additional layers can be used. The scalable encoder 100 takes two input 
sequences and generates two bit streams for miiltiplexing at a mux 140. Specifically, the 

25 input base video stream or layer is processed at a base layer encoder 110, and upsampled 
at a midprocessor 120 to provide a reference image for predictive coding of the input 
enhanced video stream or layer at an enhancement layer encoder 130. 

Note that coding and decoding of the base layer operate exactly as in the non- 
scalable, single layer case. In addition to the input enhanced video, the enhancement layer 

30 encoder uses information about the base layer provided by the midprocessor to efficiently 
code the enhancement layer. After commimication across a channel, which can be, e.g., a 
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computer network such as the Internet, or a broadband communication channel such as a 
cable television network, the total bit stream is demultiplexed at a demux 150, and the 
scalable decoder 160 simply inverts the operations of the scalable encoder 100 using a 
base layer decoder 170, a processor 180, and an enhancement layer decoder 190. 
5 The MPEG standard refers to the processing of hierarchical ordered bit stream 

layers in terms of "scalability". One form of MPEG scalability, termed "spatial 
scalability" permits data in different layers to have different frame sizes, frame rates and 
chrominance coding. Another form of MPEG scalability, termed "temporal scalability" 
permits the data in different layers to have different frame rates, but requires identical 
10 frame size and chrominance coding. In addition, "temporal scalability" permits an 
enhancement layer to contain data formed by motion dependent predictions, whereas 
"spatial scalability" does not. These types of scalability, and a further type termed "SNR 
scalability", (SNR is Signal to Noise Ratio) are further defined in section 3 of the MPEG 
standard. 

IS FIG. 2 illustrates a spatial scalable video encoder 200 according to one 

embodiment of tiie invention. The depicted encoding system 200 accomplishes layer 
compression, whereby a portion of the chaimel is used for providing a low resolution 
base layer (BS) and the remaining portion is used for transmitting edge enhancement 
information (ES), whereby the two signals may be recombined to bring the system up to 

20 high-resolution. A high resolution (Hi-Res) video input signal is split by splitter 202 
whereby the data is sent, in one direction, to a low pass filter (LPF) & downscaler 204 
and, in another direction, to a subtraction circuit 206. The low pass filter & downscaler 
204 reduces the resolution of the video data, which is then fed to a base encoder 208. In 
general, low pass filters and encoders are well known in the art and are not described in 

25 detail herein. The base encoder 208 produces a lower resolution base stream BS which is 
one input of multiplexer 240. 

The output of the base encoder 208 is also fed to a decoder 212 within the system 
200. From there, the decoded signal is fed into an interpolate and upsample circuit 214. 
In general, the interpolate and upsample circuit 214 reconstructs the filtered out 

30 resolution from the decoded video stream and provides a video data stream having the 
same resolution as the high-resolution input However, because of the filtering and the 
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losses resulting from the encoding and decoding, loss of information is present in the 
reconstructed stream. The loss is determined in the subtraction circuit 206 by subtractmg 
the reconstructed high-resolution stream from the origmal, unmodified high-resolution 
stream. The output of the subtraction ckcuit 206 is fed into a modification unit 207. The 
modification unit 207 transforms the residual signal into a signal wifli the same signal 
level range as a normal input video signal as used for video compression. The 
modification unit 207 adds a DC-offset value 209 to the residual signal. The 
modification unit 207 also comprises a clip fimction which prevents the output of the 
modification unit from going below a predetermined value and above another 
predetermined value. This DC-ofi6et and clipping operation allows the use of existmg 
standards, e.g., MPEG, for the enhancement encoder where the pixel values are in a 
predetermined range, e.g., 0. ..255. The residual signal is normally concentrated around 
zero. By adding a DC-ofifset value 209, the concentration of samples can be shifted to the 
middle of the range, e.g., 128 for 8 bit video samples. It is noted that to allow for the use 
of generic MPEG-2/4/AVC decoders at each layer instead of decoders specifically 
designed for scalable systems, a DC-ofifeet value is applied prior to encoding and 
subsequent to decoding. 

With continued reference to FIG. 2, the transformed residual signal from the 
modification unit 207 is fed to an enhancement encoder 216 which outputs a reasonable 
quality enhancement stream ES which represents a further input of multiplexer 240. 

A key feature of the invention is represented by a third input supplied to 
multiplexer 240. The third input comprises signaling information 220 embodied as a 
parameter list which is transmitted along with the MPEG-2/4/AVC compliant stream 250. 
The parameter list independently defines for each layer, how the particular layer is to be 
decoded. 

In one embodiment, the parameter list 220 includes additional signaling 
information embodied as parameter values to instruct the decoder on how to properly 
combine the various layers (e.g., BS, ES) at the decoder into a single decoded bit stream. 

The parameter values may define, for example: 

(1) a horizontal and vertical scaling factor to be applied to each layer (e.g., scale- 
up, scale-down, no scaling) 
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(2) DC compression to be applied (if any) to each layer 

(3) the stream type (e.g., MPEG-2, MPEG-4, AVC, etc.), 

(4) the FIR coeflBcients associated with the scalmg, (the more complex you make 
the FIR filter, the more perfect the scaling. It is noted that better results are achieved if 

S the decoder knows which coefficients were used in the encoder and 

(5) constant gains in the sub-band. 

(6) an identifier for a reference layer to be combined with a current layer; 

(7) how a current layer is to be combined with a reference layer; 

(8) whether a corresponding layer contains one of an interlaced or progressive 
10 video stream. 



As shown, the parameter list 220 (i.e., signaling information) is multiplexed along 
with the encoded signal for each layer (BS, ES) to allow the decoder to interpret the 
parameter values and decode the MPEG 2/4/AVC stream 250 accordingly. 
15 It should be appreciated that while the encoder 200 of FIG. 2 illustrates a two- 

layer system, the invention has broader applicability to higher order (additional) 
enhancement layers. 

It is noted that, to achieve the objective of a simple and straightforward concept 

for layering, a number of constraints are applied: 

20 • each layer has the same temporal resolution; 

• each layer codes the same picture area, but the resolution in each layer may differ. 

It is further noted that in accordance wi& the method of the invention for providing 
heterogeneous layered video support, the at least two layers (BS, ES) may be transmitted, 

25 in one embodiment, over Internet Protocol using real-time transport protocol (RTP) in a 
transmission session for each layer. While, the signaling information (220) is transmitted 
within the context of the transmission session either in-band or out-of-band within the 
transmission session. The signaling information could, for example, be transmitted using 
session description protocol (SDP). 

30 In accordance with another embodiment, the at least two layers (BS, ES) may be 

transmitted over at least one of an MPEG-2 transport stream, an MPEG-2 program stream 
and an Internet Protocol (IP) stream to the decoder, and the signaling information could 
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similarly be transmitted over at least one of an MPEG-2 transport stream, an MPEG-2 
program stream and an Internet Protocol (IP) stream to the decoder. 

In order to implement the functionality described herein, it is proposed that an 
amendment to the MPEG-2 standard is required. The following describes the details of 
5 Ihe proposed amendment. The details of the proposed amendment are disclosed as: (T) 
amendments to the stream type assignments of the MPEG-2 standard, and (II) 
amendments to the program and program element descriptors of the MPBG-2 standard. 

I Added: the differential video stream descriptor 

10 The differential video stream descriptor specifies tiie coding format of the 

associated stream as well as the applied DC oflEset For each differentially coded video 
stream carried in an ITU-T Rec. H.222.0 ISO/IEC 13818-1 stream (i.e., the document 
number of the MPEG-2 system standard), the differential video stream descriptor shall be 
included in the PMT (Program Map Table) or in the PSM (Program Stream Map), if 

IS PSM is present in the program stream. 



Table L Fields of differential video stream descriptor 



Syntax 


No. of bits 


Mnemonic 


Differential video stream descriptorQ { 






descriptor_tag 


8 


uimsbf 


descriptor_length 


8 


uimsbf 


stream_type 


8 


uimsbf 


DC offset 


16 


uimsbf 


> 







20 Semantic definition of fields of Table I: 

(a) stream type - An 8 bit unsigned integer that specifies the encoded format of the 
associated differential video stream, encoded as specified in table 2-29 of ITU-T Rec. 
H.222.0 ISO/IEC 13818-1. Streamjype values that mdicate other than video streams are 
25 forbidden. Also a stream type value of OxlC is forbidden. 
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(b) DC offset - A 16 bit xmsigned integer that specifies the DC offset that shall be applied 
on the decoded signal when reconstructing the video output, 
n. Added: The Spatially layered video stream descriptor 

The spatially layered video stream descriptor specifies for a video stream in a 
5 layered video system, the layer, the exact horizontal and vertical re-sampling factors, and 
the recommended filter coefficients for the horizontal and vertical re-sampling, as 
specified in 2-15. The spatially layered video stream descriptor shall be associated to 
each video stream, hence to each base and each enhancement stream, in a layered video 
system. For each such stream carried in an ITU-T Rec, H.222.0 | ISO/ffiC 13818-1 
10 stream , the spatially layered video stream descriptor shall be included in the PMT or in 
the PSM, if PSM is present in the program stream. 



Table n. Fields of spatially layered video stream descriptor 



Syntax 


No. of bits 


Mnemonic 


Spatially layered video stream descriptorQ { 
descriptor_tag 
descriptor_length 
layer 

reference_layer 
referenced_flag 
reserved 

if (referenced_flag='0') || ((referenced_flag='r) && 

(reference_layei>0)){ 

up_horizontal 
down_horizontal 
up_vertical 
down_vertical 

nuraber_of^horizontal_coefBcients 
number__of_yertical_coefl5cients 

for (i=0; 

i<iiiimber_of_horizontal__coefficients;i-H-){ 
hor fir(i) 

} 

for (i=0; i<3iumber_of_vertical_coefBcients;i++){ 
ver fir(i) 

} 

} 

> 


8 
8 
4 
4 
1 
7 

4 
4 
4 
4 
4 
4 

16 
16 


uimsbf 

uimsbf 

uimsbf 

uimsbf 

bslbf 

bslbf 

uimsbf 
uimsbf 
uimsbf 
uimsbf 
uimsbf 
uimsbf 

uimsbf 
uimsbf 
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Semantic definition of fields of Table H: 

(a) layer -A 4 bit unsigned integer that specifies the index number of the layer of the 
associated video stream. 

(b) reference layer - A 4 bit unsigned integer that identifies the index number of the 
layer of the yideo stream with the spatial resolution to which this video stream is re- 
sampled. For example, a reference layer value of 0 indicates that this video stream is not 
re-sampled. 

(c) referenced flag - A one bit flag that, if set to ' 1% indicates that this video stream has a 
spatial resolution to which one or more other streams are re-sampled. 

If the referenced_flag is set to '0% then this descriptor contains filter information 
for the re-sampling to the resolution of video stream referenced by the reference Jayer 
field. 

If the referenced_flag is set to '0', then the preceding referenced Jayer field shall 
be coded with a value larger than zero. 

If the referenced_flag is set to while the preceding reference Jayer field is 
coded with a value larger than zero, then this descriptor contains filter information for the 
next stage re-sampling of the intermediate re-sample result at the spatial resolution of this 
stream to the resolution of video stream referenced by the reference Jayer field. 

(d) up horizontal, down horizontal - Two 4 bit unsigned integers specifying that the 
horizontal re-samplmg fector shall be equal to (up_horizontal) / (down^horizontal). A re- 
sampling factor larger than 1 (for example 8/3) indicates up-sampling, a factor smaller 
than 1 down-sampling. For both fields a value of zero is forbidden, 

(e) up verticaL down vertical - Two 4 bit unsigned integers that specify that the vertical 
re-sampling factor shall be equal to (up_vertical) / (down_vertical). A re-sampling fector 
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larger than 1 (for example 8/3) indicates up-sampling, a factor smaller than 1 down- 
sampling. For both fields a value of zero is forbidden. 

(f) number of horizontal cneffirifintQ - A 4 bit unsigned integer that specifies Ihe 
number of horizontal filter coefficients in this descriptor. 

fe) number of vertical coefficients - A 4 bit unsigned integer that specifies the number 
of vertical filter coefficients in this descriptor. 

(h) hor _ fir(i) - A 16 bit unsigned integer that specifies the horizontal FIR filter 
coefficient with index i. The central coefficient has index value zero. 

By defining the above signaling parameters per layer, a high degree of flexibilily 
is achieved. Particularly, in the prior art it is a requirement that the base layer exist at flie 
lowest resolution. In the present scheme, no such limitation exists. The aforementioned 
parameters may be independently defined for each layer, independent of any other layer. 

Another feature of the invention is the case where multiple enhancement layers 
are defined. In this case, a separate parameter list could be constructed to define a 
multiplicity of quality levels. For example, for a four layer video stream including a base 
layer and three enhancement layers, a first parameter list could be constructed to define a 
combination of the base layer BS with both enhancement layers ESI and ES2. A second 
parameter list could be constructed to define a combination of the base layer BS with the 
second and fourth enhancement layers (BS + ES2 + ES4). Other combinations should be 
apparent to the reader. All of the combinations of interest to a user may be 
simultaneously transmitted as elements of parameter list 220. 

FIG. 3 illustrates a decoder 300 according to one embodiment of the invention. 
Figure 3 illustrates a decoder for decoding the encoded signals processed by the layered 
encoder 200 of FIG. 2. The base stream BS is decoded in base decoder 302 in accordance 
with those parameters fi-om parameter list 200 which are associated with the base layer 
BS. The decoded output Jfrom the decoder 302 is upconvCTted by an upconverter 306 and 
then supplied to an addition unit 310. The enhancement stream ES is decoded in a 
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decoder 304 in accordance with those parameters from parameter list 200 which are 
associated with the enhancement stream ES. The modification unit 308 performs the 
inverse operation of the modification unit 207 in the encoder 200. The modification unit 
308 converts the decoded enhancement stream from a normal video signal range to the 
5 signal range of the original residual signal. The output of the modification unit 208 is 
supplied to the addition unit 310, where it is combined with the output of the upconverter 
306 to form the output of the decoder 300. 

EXAMPLES: 

10 

Example 1- A dual layer configuration utilizing an AVC decoder in the base layer and an 
MPEG-2 decoder in the enhancement layer. : 

Referring to FIG. 4, Tables I and n define a parameter list 220 that would be 
15 broadcast over a communication channel as supplemental information to inform a 

decoder as to how to combine the various streams (e.g., Layl, Lay2) to output a single 
decoded video stream. 

Referring to the first row of the parameter list, (i.e., the row describing parameters 
specific to the base layer, Layl) the encoder side parameter list instructs the decoder to 
20 use an AVC decoder in the base layer (Layl). Next, the parameter list instructs the 
decoder that the DC offset parameter is zero. This instructs the decoder 300 not to 
subtract a DC offset in the base layer prior to combining this layer with the enhancement 
layer, Lay2. The next four columns of the first row are labeled upH, dwH, upV and dwV, 
respectively, and refer to an upscaling factor in the horizontal (upH), downscaling factor 
25 in the horizontal (dwH), an upscaling factor in the vertical (upV) and a downscaling 

factor in the vertical (dwV). The decoder 300 uses these parameter in pairs. That is, the 
decoder 300 takes a ratio of the first two parameters, upH/dwH to determine whether the 
horizontal is to be upscaled, downscaled or not scaled at all. In the present example, the 
horizontal scaling ratio 

30 



Hor. Scaling ratio = upH/dwH =2/1=2 
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Similarly, for the vertical direction, the decoder 300 takes a ratio of upV/dwV to 
detennine whether the vertical is to be upscaled, downscaled or not scaled at all. In the 
present example, the vertical scaling ratio 

5 

Ver, Scaling ratio = upV/dwV =2/1= 2 (2) 

After performing any DC ofi&ets and adjusting for the appropriate horizontal and vertical 

offsets, the next column refers to what layer the previous layer is to be added to. After 
10 performing the operations described on the base layer (Layl) the result is combined with 

the single enhancement layer, Lay2. 

Table I provides a number of parameters specific to the enhancement layer, Lay2. 

Specifically, the parameter list instructs the decoder to use an MPEG-2 decoder for the 

single enhancement layer, Lay2. The parameter list further instructs tibie decoder to 
15 perform a DC offset of 128. The (recommended) filter coefficients for performing this 

offset are defined in Table 11. Specifically, seven filter coefficients are defined in both 

the horizontal and vertical direction. 

Example 2 - Three layer configuration utilizing an AVC decoder in the base layer (Lay 
20 1) and both enhancement layers (Lay2, Lay3). 

Referring now to FIG. 5, Tables I and U define a parameter list 220 that would be 
broadcast over a commimication channel as supplemental information to inform a 
decoder as to how to combine the various streams to output a single decoded video 
25 stream. 

Referring to the first row of Table I of the parameter Ust, the parameter Ust 
instructs the decoder to use an AVC decoder in the base layer (Layl). The parameter Ust 
further instructs the decoder that the DC offset parameter is zero. This instructs the 
decoder 300 not to subtract a DC offset in the base layer prior to combining this layer 
30 with the first enhancement layer, Lay2. In the present example, the horizontal scaling 
ratio is 2 and the vertical scaling ratio is also 2. The next column refers to what layer the 
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base layer, Layl, is to be added to. In this case. Lay 1 is to be added to Lay2, the first 
enhancement layer. Both enhancement layers, i.e., Lay2 and Lay3 have similar 
parameter values defining DC offsets of 128 and no scaling in both the horizontal and 
vertical directions. 

5 

Example 3 - Three layer configuration utilizing an AVC decoder in the base layer and 
both enhancement layers. Each layer added in a parallel configuration. 

Referring to FIGS. 6 and 7, Tables I and II of FIG. 6 define a parameter list 220 

10 that would be broadcast over a commxmication channel as supplemental information to 
inform a decoder as to how to combine the various streams (i.e., Layl, Lay2, LayS) to 
output a single decoded video stream. 

Referring to the first row of Table I of the parameter list of FIG. 6, the parameter 
list instructs the decoder to use an AVC decoder in the base layer (Layl), The parameter 

15 list further instructs the decoder that tiie DC offset parameter is zero. This instructs the 
decoder 300 not to subtract a DC offset in the base layer prior to combining this layer 
with the first enhancement layer, Lay2. In the present example, the horizontal scaling 
ratio is calculated as 2 and the vertical scaling ratio is calculated as 2. The next column 
"Reference Layer (scaling)" refers to which layer the base layer, Layl, is to be added to 

20 next In this case, Layl is to be added to Lay2, the first enhancement layer. The next 
column, ''Reference flag" defines a parameter value for instructing the decoder on the 
order in which any required DC compensation and scaling is to be performed for the 
present layer (Layl) prior to summing it with the layer defined by the Reference flag 
parameter. In the instant example, Layl requires no DC compensation, however a 

25 "Reference Flag" parameter value of one (1) instructs the decoder to perform any 

required scaling, which in the instant case is 4/1, prior to sunmiing Layl with Lay2, via 
summation block 72 of Fig. 7. 

Continuing with &e instant example, referring now to Lay2, the first enhancement 
layer, the '^Reference Flag" parameter value of zero (0) as before, instructs the decoder 

30 to apply any required DC compensation and scaling to Lay2 prior to simmiing Lay2 with 
Lay3. 
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Rxam ple 4 - Three layer configuration utilizing an AVC decoder in the base layer and 
both enhancement layers. 

5 Referring to FIGS. 8 and 9, Tables I and 11 of FIG. 8 define a parameter list 220 

that would be broadcast over a communication channel as supplemental information to 
inform a decoder as to how to combine the various streams (i.e.. Lay 1, Lay2, Lay3) to 
output a single decoded video stream. 

Referring to the first row of Table I of the parameter list, the encoder side 

10 parameter list instructs the decoder to use an AVC decoder in the base layer (Layl). The 
parameter list further instructs the decoder that the DC offeet parameter is zero. This 
instructs the decoder 300 not to subtract a DC offset in the base layer prior to combining 
this layer with the first enhancement layer, Lay2. In the present example, the horizontal 
scaling ratio is calculated as 2 and the vertical scaling ratio is calculated as 2. The next 

15 colunm "Reference Layer (scaling)" refers to which layer the base layer, Layl, is to be 
added to next. In this case, Layl is to be added to Lay2, the first enhancement layer. The 
next colunm, '^Reference flag*' defines a parameter value for instructing the decoder to 
perform any required DC compensation and scaling for the present layer (Layl) prior to 
summing it with the layer defined by the Reference flag parameter. In the instant 

20 example, Layl requires no DC compensation, however and a 4/1 scaling prior to 
summing it with Lay2, the first enhancement layer. 

Continumg with the instant example, referring now to Lay2, the *TReference Flag** 
parameter value of one (1) instructs the decoder to apply any required DC compensation 
to the present layer as before. However, in this case, the value of one (1) instructs the 

25 decoder to apply scaling after the present layer is summed with the previous layer. In the 
instant example, a DC compensation of 128 is performed for Lay2, followed by a 
summation with Lay 1, via summation block 92 of FIG. 9, followed by a 2/1 scaling of 
the output of the output of summation block 92 of FIG. 9. 

Continuing with the mstant example, referring now to Lay3, the second 

30 enhancement layer, the **Reference Flag" parameter value of one (1) once again instructs 
the decoder to apply any required DC compensation to the present layer as before, which 
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for the present layer is a DC compensation of magnitude 128, identical to that applied to 
the previous layer. Because the scaling factor for the present layer if one (1), there is no 
scaling block shown to the right of summation block 94 of FIG. 9. 

Although this invention has been described with reference to particular 
5 embodiments, it should be appreciated that many variations can be resorted to without 
departing jfrom the spirit and scope of this invention as set forth in the appended claims. 
The specification and drawings are accordingly to be regarded in an illustrative manner 
and are not intended to limit the scope of the appended claims. 



